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DETAILED ACTION 

1. This Office action for U.S. Patent Application 10/802,428 is responsive to 
communications filed 04 May 2009, in reply to the Non-Final Rejection of 04 February 
2009. Currently, Claims 1-3, 5, 7-15, 17, 19-25, and 27 are pending. 

2. In the previous Office action, Claims 1-3, 5-12, and 25 were rejected under 35 
U.S.C. 101 as non-statutory. Claims 1-3, 5-7, 10-15, 17-19, and 22-25 were rejected 
under 35 U.S.C. 103(a) as obvious over "Motion-based Segmentation Using a 
Thresholded Merging Strategy on Watershed Segments" (de Smet et al.) in view of "K- 
Harmonic Means-A Data Clustering Algorithm" (Zhang et al.). Claims 8, 9, 20, and 21 
were rejected under 35 U.S.C. 103(a) as obvious over de Smet et al. in view of Zhang 
et al. and in view of "A Video Segmentation Algorithm for Hierarchical Object 
Representations and its Implementation (Herrmann et al.). Claims 27 and 29 were 
rejected under 35 U.S.C. 103(a) as obvious over de Smet et al. in view of Zhang et al. 
and in view of U.S. Patent 6,084,91 2 A (Reitmeier et al.). 

Response to Amendment 

3. Applicant's amendments to the specification have been fully considered. The 
rejection of Claims 1-3, 5, and 7-12 under 35 U.S.C. 101 is withdrawn. 

4. Applicant's amendments to the claims have been fully considered. The rejection 
of Claim 25 under 35 U.S.C. 1 01 is withdrawn. 
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Response to Arguments 

5. Applicant's arguments filed with respect to claim 1 have been fully considered but 
they are not persuasive. Applicant states that the de Smet and Zhang references do 
not disclose the claimed limitations of recalculating K regression functions or stopping 
the repeated algorithm according to "membership probabilities". 

Regarding the alleged deficiency of the references to show calculating the K 
regression function based on the membership probability, it is noted that in the Final 
Rejection of 29 May 2008, the probability function was explicitly mapped with the 
function q(i,k) in the Zhang reference (pg. 4). Each of the previous three Office actions 
made reference to "membership probability q(i,k)" in discussing Zhang et al. Applicant 
has not provided any evidence to the contrary, in support of the conclusory statements 
in page 11 of the arguments that "there is nothing in. ..the KHM clustering described in 
Zhang" to recalculate the K regression functions based on membership probabilities. 
Applicant is reminded that once a prima facie case of patentability has been 
established, the burden shifts to applicant to come forward with rebuttal evidence or 
arguments to overcome the prima facie case. In re Bell, 991 F.2d 781, 783-784, 26 
USPQ2d 1529, 1531 (Fed. Cir. 1993); In re Rijckaert, 9 F.3d 1531, 1532, 28 USPQ2d 
1955, 1956; (Fed. Cir. 1993); In re Oetiker, 977 F.2d 1443, 1445, 24 USPQ2d 1443, 
1444 (Fed. Cir. 1992). 

Next, assuming that q(i,k) in Zhang et al. is a calculation of probability of 
membership for each data point, it must be shown that this calculation is used "to 
recalculate regression functions", and that this recalculation stops if changes in 
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membership probabilities or changes in the K-regression function satisfy a stopping 
criterion. 

As previously stated in page 5 of the Non-Final Rejection of 04 February 2009, in 
Zhang et al., m(k) is a regression function according to the normal definition of the term. 
The calculation of the center point is performed is performed for K center points (Zhang, 
pg. 4: "the center positions, m(k), k=1,...,K"), so certainly the calculation of m(k) for 
each of these center points is a process of calculating "K regression functions". The 
description of m(k) in Zhang et al. shows it as a "recursive formula" in which after the 
initial positions are calculated, "the new positions of the centers are calculated" (pg. 5). 
Since m(k) incorporates q(i,k), the recalculation of m(k) at each iteration is "at least 
based on" q(i,k). 

In addition, the recursion in Zhang is described as continuing "until the 
performance value stabilizes". Here, "stabilize" is assumed to have its ordinary 
definition of being held to a steady state, or constant. This stabilization is shown 
graphically in pages 15, 18, 21, and 24, in which the positions of the center points only 
change minimally after enough iterations. Then, this convergence of the center points 
satisfies the test of "determining whether changes in the K regression function satisfy a 
stopping criterion", one of the two-pronged test given in claim 1 . 

Considering this, all rejections of the claims over the prior art are maintained. 
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Claim Rejections - 35 USC § 103 

6. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

7. Claims 1-3, 5, 7, 10-15, 17, 19, and 22-25 are rejected under 35 U.S.C. 103(a) 
as being unpatentable over "Motion-based Segmentation Using a Thresholded Merging 
Strategy on Watershed Segments" (de Smet et al.) in view of "K-Harmonic Means-A 
Data Clustering Algorithm" (Zhang et al.). De Smet et al. discloses using an iterative 
segment-merging technique to determine information for an image (abstract). 

Regarding claims 1,13, and 25, in de Smet et al., an initial motion field is first 
determined with a block-matching technique on 4 x 4 blocks (§ 2.1). These initial block 
motion vectors are used for the initial segmentation (§ 2.3). Then, the step of 
performing the block-based motion estimation is the claimed step of "providing data 
points". Next, the segments are iteratively merged according to similar or shared 
motion, according to the K-means clustering algorithm (§ 2.3). This is the claimed step 
of "clustering the data points". When this process is finished, the result is a series of 
large segments corresponding to distinct moving regions of an image, each with an 
associated motion vector (§ 2.3). This is the claimed step of "providing motion 
estimation". However, the present invention specifies performing regression clustering 
according to a K-Harmonic Means function, which is not the same as the K-means 
function of de Smet et al. 
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Zhang et al. discloses the K-Harmonic Means data clustering algorithm. 
Regarding claim 1, 3, and 25, Zhang et al. teaches selecting K centers m(l) from N data 
points x(i) (pg. 1), initializing center points (pg. 2) and performing an initial iteration (pg. 
5), calculating distance d(i,l) between data point x(i) and center point m(l) (pg. 4), 
calculating membership probability q(i,k) and weighting function p(i,k) (pg. 5), 
recursively calculating new m(k), (pg. 5), and stopping when the recursively-calculated 
performance value stabilizes, that is, when its change with each iteration becomes small 
(pg. 5). Then, the K calculations of centers m(l) are the claimed "regression functions" 
for performing regression clustering according to the K-harmonic means function. The 
calculation of each iteration of the recursive function is the claimed recalculation based 
on the membership probability, as m(k) is dependent on probability function q(i,k). The 
stabilization is the claimed "stopping criterion". 

De Smet et al. discloses the claimed invention, except for using K-Harmonic 
Means function to perform regression clustering. Zhang et al. teaches that it was known 
to perform data clustering with the K-Harmonic means function. Therefore, it would 
have been obvious to one having ordinary skill in the art at the time the invention was 
made to perform clustering based on a K-Harmonic means function, as taught by Zhang 
et al., rather than a linear function such as the K-means function of de Smet et al., since 
Zhang et al. states in the abstract that K-harmonic clustering is less sensitive to 
detrimental effects from sub-optimal initialization than conventional clustering 
techniques. Additionally, the K-harmonic means function was specifically designed as 
an improved version of the K-means function, which "significantly improves the quality 
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of clustering results" compared with K-means (abstract). Then, the de Smet et al. 
algorithm contains a basic method which the present invention is an improvement 
thereof. The prior art Zhang et al. describes a known technique, the K-Harmonic means 
clustering algorithm, applicable to the base segmentation method of de Smet et al., by 
substituting it for the K-means algorithm. Then, one having ordinary skill in the art 
would have recognized that applying the K-harmonic means algorithm of Zhang et al. to 
de Smet et al. would have yielded the predictable result of "significantly [improved] 
quality of clustering results" and resulted in an improved system. Therefore, it is 
respectfully submitted that the use of K-harmonic clustering in de Smet et al. is 
considered obvious, since it has been held that applying a known technique to a known 
method ready for improvement to yield predictable results involves only routine skill in 
the art. Dann v. Johnston, 425 U.S. 219, 230, 189 USPQ 257, 261 (1976); In re Nillsen, 
851 F.2d 1401, 1403, 7 USPQ2d 1500, 1502 (Fed. Cir. 1988). 

Regarding claims 2, 3, 14, and 15, as previously mentioned, de Smet et al. 
produces a motion vector for each segment in an image (§ 2.3). As a result, the most 
important moving areas are determined (§ 3). Regarding claims 10 and 22, in de Smet 
et al., pixels are set as (x,y,t) triples, with x and y as spatial coordinates and t as a time 
coordinate (§ 2.2). 

Regarding claims 11 and 23, de Smet et al. illustrates motion fields (figures 3-6). 
Although these motion fields are not shown as overlaid on the images, the examiner 
takes Official Notice that it was well-known in the art at the time of the invention to 
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display a motion field superimposed on an image to provide a visual representation of 
motion vectors. 

Regarding claims 12 and 24, de Smet et al. illustrates highlighted motion 
segments overlaid on an image (figures 11 and 12). 

Regarding claims 5 and 17, in Zhang et al., a clustering in which initialization is 
randomized is described (pg. 11). 

Regarding claims 7 and 19, insensitivity to initialization is an inherent result of the 
K-Harmonic means algorithm (abstract). 

8. Claims 8, 9, 20, and 21 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over de Smet et al. in view of Zhang et al. as applied to claims 1 and 13 
above, and further in view of "A Video Segmentation Algorithm for Hierarchical Object 
Representations and its Implementation" (Herrmann et al.). Claims 8, 9, 20, and 21 
disclose using color information to segment images, but de Smet et al. only discloses 
"standard watershed techniques" (§ 2.2) to perform the initial segmentation without 
providing details. 

Herrmann et al. discloses a method for image segmentation to extract objects 
from a moving image. Regarding claims 8, 9, 20, and 21, after an initial block-matching 
motion estimation, similar to de Smet et al., images are segmented according to specific 
color information, followed by shape analysis, and lastly motion analysis to merge 
regions to determine objects (§ II). This color, shape, and motion information form the 
claimed "predetermined criteria". In color analysis, a region is determined as 
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homogeneous if the pixel difference in the region is below a threshold. Homogenous, 
connected areas are determined as "quasi-flat zones". These quasi-flat zones are 
further processed and become the basis for further segmentation (§ II. B). Then, the 
color analysis is the claimed step of "portioning data according to color". 

De Smet et al., in combination with Zhang et al., disclose a majority of the 
features of claims 8, 9, 20, and 21 as discussed above, the claimed invention except for 
color segmentation. Herrmann et al. teaches that it was known to segment a moving 
image according to color. Therefore, it would have been obvious to one having ordinary 
skill in the art at the time the invention was made to determine image segmentation by 
color as taught by Herrmann et al., since Herrmann et al. teaches in page 205, third 
paragraph, that color analysis produces the most accurate type of segmentation. 

9. Claims 27 is rejected under 35 U.S.C. 103(a) as being unpatentable over de 
Smet et al. in view of Zhang et al., and in view of US Patent 6,084,912 A (Reitmeier et 
al.) Claim 27 is in means-plus-function format, complying with 35 U.S.C. 112, sixth 
paragraph. Accordingly, the "system" of claim 27 will be limited to a general-purpose 
computer such as a PC, as illustrated in figure 5 of the specification of the present 
invention. Although it is implied that the algorithms of de Smet et al. and Zhang et al. 
are computer-operated, neither de Smet et al. nor Zhang et al. explicitly teach this. 

Reitmeier et al. discloses a video encoder. This encoder may operate on 
MPEG-4 video (column 1: line 57), as specified in paragraphs [0006] and [0007] of the 
present invention as a codec on which the present invention is applied. Regarding 
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claim 27, the encoder of Reitmeier et al. may operate as a software application on a 
general-purpose computer (column 2: lines 64-67). 

De Smet et al., combined with Zhang et al., discloses the claimed invention 
except for encoding video on a general-purpose computer. Reitmeier et al. teaches that 
it was known to implement an MPEG-4 encoder as software embedded on a computer. 
Therefore, it would have been obvious to one having ordinary skill in the art at the time 
the invention was made to encode the system of de Smet et al. and Zhang et al. in a 
software MPEG encoder embedded on a computer, as taught by Reitmeier et al., in 
order to perform computationally complex functions such as motion compensation, 
quantization, and variable-length encoding inherent in the video coding process. 



Conclusion 

10. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 
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Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to David N. Werner whose telephone number is (571)272- 
9662. The examiner can normally be reached on Monday-Friday from 10:00-6:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehrdad Dastouri can be reached on (571) 272-7418. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
ID. N. W./ 

Examiner, Art Unit 2621 
/Dave Czekaj/ 

Primary Examiner, Art Unit 2621 



